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DEVICE FOR CLiASSIFYING AT LEAST ONE OBJECT IN THE SURROUNDING 

FIELD OF A VEHICLE 

Background Information 

The present invention is based on a device for classifying at 
least one object in the surrounding field of a vehicle 
according to the definition of the species in the independent 
claim. 

A camera-based precrash detection system is known from DE 100 
25 678 Al • In this context, potential opposing parties in an 
accident are marked in the image data of a monoscopic camera, 
and their distance and relative velocity are subsequently 
determined, so that objects specifically putting one at risk 
can be subjected to classification by type in real time. This 
method identifies regions within a camera image, in which 
there are highway users or obstacles. As described above, the 
identification is carried out with the aid of a classifier 
specially trained for the highway users and obstacles to 
detect. The corresponding, identified regions are then marked 
and measured for their distance from the observer, using a 
distance -measuring sensor. Finally, the selected regions are 
supplied to a type-classification unit for exactly determining 
the type of highway user or obstacle. 

Summary of the Invention 

The device of the present invention for classifying at least 
one object in a surrounding field of a vehicle, possessing the 
features of the independent claim, has the advantage over the 
related art that the object is classified on the basis of at 
least its three-dimensional shape and its dimensions. In this 
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context, the environmental sensor system is configured to be 
able to ascertain the shape and the dimensions. The shape and 
the dimensions- are important parameters for clearly- 
identifying potential opposing parties in an accident. 
5 Protective measures or other vehicle functions may be 
activated as a function of these parameters. The more 
effective classification then allows this activation to be 
carried out more accurately. 

10 Advantageous improvements of the device, which relates to the 
independent claim and is for classifying at least one object 
in the surrounding field of a vehicle with the aid of an 
environmental sensor system, are rendered possible by the 
measures and further refinements introduced in the dependent 

15 claims. 

It is particularly advantageous that the device additionally 
takes the velocity of the at least one object into 
consideration during the classification. Clear 

2 0 identification may then take place by viewing the shape, 
dimensions, and velocity of the object as a whole. Then, 
using data from a data base, a classification validity check 
as to whether the shapes and dimensions in question, 
together with the respective velocity, can even characterize 

25 a known object, is carried out on the basis of these 

parameters. In addition, the classification may be carried 
out by determining the direction of the object, i.e. its 
orientation. This determines the assessment of the 
dimensions of the object, for the visible dimensions of the 

30 object are determined as a function of the orientation of 
the object with respect to an observer. 

In addition, it is advantageous that the device is coupled to 
at least one vehicle system, so that the device controls the 
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at least one vehicle system as a function of the 
classification. In the case of, e.g. protective measures, 
such as active operational intervention, such 
classifications may now be taken into consideration, in 
5 order to consider the behavior of such classified objects 
for the type of intervention. 

The vehicle system may advantageously be a protection system, 
such as ESP, a brake assist system, or an occupant -restraint 
10 system, to which, for example, a pedestrian protection 

system may also belong. However, the vehicle system may also 
be a driver assistance system, such as a lane-change warning 
device, when it is determined on the basis of the object 
classification that a lane-change event is probable. 

15 

The environmental sensor system may have a stereoscopic 
camera for measuring the dimensions and the three- 
dimensional shape. However, a monoscopic camera having 
moving stereo is also possible. Further alternatives include 
20 LIDAR, scanning radar sensors, or other stereo-capable 
sensors . 

Brief Description of the Drawing 

25 Exemplary embodiments of the present invention are shown in 
the drawing and are explained in detail in the subsequent 
description . 

The figures show: 

30 

Fig. 1 a block diagram of the device according to the present 
invention; 

Fig- 2 a flowchart; and 
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Fig. 3 an example of classification. 
Description 

5 

Present mass-produced systems for environmental sensing 
include, for example, parking-assistance systems, which detect 
the distance to an object with the aid of ultrasonic sensors 
and emit a warning when the measured distance becomes too 

10 small. Also mass-produced is so-called adaptive cruise control 
(ACC) , which determines spacing and velocity with the aid of a 
radar technology. This allows the functionality of a vehicle- 
speed controller, which holds the set speed constant, to be 
expanded. In this manner, a desired safety distance may be' 

15 additionally maintained while controlling the set speed. 

At present, systems for environmental detection are being 
developed, using sensors such as video, radar, near and far 
infrared, ultrasonic, or LIDAR sensors, or range -image 

20 cameras, as well as a combination of these sensors. With the 
aid of these systems, objects should be detected with higher 
reliability than up to this point and new uses should be 
developed- These include, for example, an object -detection 
system, in which position measurements, motion measurements, 

25 and object measurements are ascertained, or a traffic-lane, 
detection system, in order to allow, for example, a lane- 
change warning. Such systems are developed for use in safety 
and luxury applications. 

30 Object classification is an additional important subject. 
Classification is the dividing-up of the objects into 
predefined classes. In particular, safety and comfort systems 
of the vehicle may be improved with the aid of this additional 
information about the detected objects. In particular, in 
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critical situations such as accident scenarios, safety systems 
may be used in a more differentiated and effective manner with 
the aid of object classification. 

5 1- Possible applications of a classification of objects in the 
surrounding field of a vehicle include: 
safety functions: 

- protection of vehicle occupants 

- partner protection, that is, the protection of other highway 
10 users 

- improvement of the crash compatibility between parties 
involved in an accident 

- warning functions 
comfort functions ; 

15 - expansion of the ACC functionality 

Reliable descriptions of the surrounding field of a vehicle 
are necessary for many future safety functions. Interventions 
in the vehicle control without the activation of restraining 

2 0 devices could be damaging than useful, when they are executed 
by mistake. In devices such as ACC, data regarding relevant 
objects are acquired, for example, from radar sensors and are 
mostly limited to geometric information, such as position and 
velocity. Further information about the type of object is not 

25 available. However, a detailed description of the surrounding 
field of the vehicle and higher reliability of the information 
are necessary for eliminating instances of false activation or 
other shortcomings of present systems. If an object is not 
only detected, but also the type of object is determined, many 

30 different measures become possible: 

An important field of application of object classification in 
the surrounding field of the vehicle is partner protection. 
For example, when an imminent collision with a pedestrian is 
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detected, a pedestrian airbag at the front of the vehicle is 
triggered or the engine hood or front opening hood is raised. 
This cushions the impact of the pedestrian with the vehicle, 
and impact with the engine block situated directly under the 
5 front opening hood is prevented. However, a pedestrian 

sensory system mostly operates, using contact sensors for 
detecting impact. In this context, there is a danger of false 
activation, because such a contact sensor could also cause 
activation, for example, when the vehicle strikes a 
10 delineator. 

Such false activation may be prevented with the aid of object 
classification. This allows repair costs to be reduced, in 
particular when activating non-reversible restraint systems 
15 such as airbags- 

The use of object classification also allows such a system to 
be activated in an anticipatory manner. Early detection of 
the imminent pedestrian collision allows the airbag to be 

20 activated without the necessity of detection by the contact 
sensor. The decision to activate on the basis of the contact 
sensor may be made more reliably, using the result of the 
classification system. Consequently, the robustness of the 
system may be increased, or contact sensors could even be 

25 dispensed with. 

In particular, in the case of so-called sport-utility 
vehicles (SUV*s), because of the raised front of the SUV, the 
danger for smaller vehicles in a collision is considerably 
3 0 higher than when the two front ends are at the same 

elevation. If an imminent collision between a small car and 
an SUV equipped with a classification system is detected with 
the aid of an object classification system, then its front 
end may be lowered, for example, with the aid of an air 
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suspension- In the event of a collision with another SUV or 
another vehicle of a similar size class, this lowering would 
not occur. Thus, with the aid of a classification system, it 
may be ensured that the greatest possible crash compatibility 
5 is always produced. This lessens the severity of the 
accident. An alternative adaptation of the crash 
compatibility may only be carried out on the basis of 
geometric object data, such as the height of the accident 
participant. However, this would not be reliable enough, 
10 since, for example, a bicyclist may have the same height as 
an SUV. The described measure is only rendered possible by a 
reliable classification system. 

The use of object classification in the monitoring of the 
15 surrounding field can be very useful for functions, such as 

automatic emergency braking or an automatic evasive maneuver. 
In this connection, it is advantageous to be able to 
distinguish between objects worth being protected, such as 
highway users, and objects less worthy of protection, such as 
20 delineators or guard rails. If an imminent, unavoidable 
collision with a pedestrian is detected, then emergency 
braking should be carried out. If the detected object is, for 
example, a delineator or a small tree, then the damage 
produced by emergency braking or an evasive maneuver, e.g. 
25 the danger to the following traffic, may be greater than the 
benefit. In the case of objects not worth protecting, such 
unintentional activation of the system acting on the vehicle 
dynamics may only be prevented with the aid of a 
classification system . 

30 

In addition, when calculating an evasive maneuver in view of 
the knowledge of the object classification, the probable 
behavior of the object to be avoided may be taken into 
account. For example, a pedestrian can make a 180® turn more 
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rapidly than a vehicle. However, a pedestrian cannot 
accelerate as rapidly in a constant direction, which means 
that knowledge of the object classification is necessary for 
reliably determining a course of avoidance . 

An additional possibility of the classification is to use it 
for checking the object input data for their validity. Due to 
measuring errors, it is possible, for example, for the width 
of a 1.65 m wide vehicle to be measured to be only 1 m, which 

10 is too small. There are presently no motor vehicles of this 
width, which means that the result would therefore be 
implausible. If the vehicle is nevertheless classified as a 
small passenger car, then the width can be checked for 
validity. The classification allows the object width to be 

15 corrected to that of the class of small passenger cars. It is 
possible that systems, such as automatic avoidance, would not 
execute any actions in the case of an object having a width 
of 1 m, although it would be necessary. In this example, the 
desired action would still be initiated on the basis of the 

20 classification result. 

In this case, the width designates the real value. Since the 
orientation of the object is known, the width may always be 
deduced from the measurement. Consequently, one then obtains 
25 the actual width of the object. 

Warning systems, such as a lane-change warning device, may 
also be assisted by an object-classification system. In such 
systems, the lane in which the reference vehicle is situated 
3 0 is detected. If the vehicle leaves this lane, e.g. the vehicle 
slowly drifts off due to fatigue symptoms of the driver, then 
a warning is emitted. Such a system may be assisted by an 
object-classification system. If the roadway-edge markings 
cannot be clearly detected due to snow, rain, dirt, or 
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deterioration, but delineators or guard rails at the edge of 
the roadway are classified, then these may be used to assist 
in estimating the path of the traffic lane. 

5 By evaluating the classification, systems may be adapted to 

the current situation of the surrounding area. A pedestrian on 
the street may be an indication that more pedestrians are to 
be expected. Increased attentiveness is necessary in this 
situation. 

10 

The workload of the vehicle operator may be reduced by comfort 
functions, so that more relaxed, and therefore safer driving 
is possible- In the case of an expanded ACC system, the 
control parameters may be adjusted automatically, using the 
15 knowledge of the class of a preceding object. For example, the 
different acceleration and deceleration capabilities of the 
preceding vehicle may be taken into account . 

Different types of sensors or combinations of many different 
20 sensors may be used for monitoring the surrounding area of the 
vehicle. Objects are detected in the monitored surrounding 
field and described, for example, by position, shape, 
dimension, and movement. According to the present invention, 
the shape and the dimensions of the object are used as the 
25 first parameter for classification. A further refinement then 
provides for the object velocity to be additionally used for 
object classification. The distance to the observer determines 
the quality of the classification: Because of the measuring 
inaccuracy and the resolution, an object a long distance away 
30 may not be classified as effectively as in the case of a short 
distance. The difference between the width of a small and a 
large passenger car is only a few dm. The measuring accuracy 
in the lateral direction at 50 m is approximately in this 
range. The resolution also plays a role here. If an object is, 
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e.g. only 1 m away, then one image pixel represents, for 
example, 1x1 cm. If the object is 50 m away, then one pixel 
represents, e.g. 50 x 50 cm. The numerical values are selected 
here for the sake of comprehension- 

5 

The object velocity may be directly measured by environmental 
sensors, such as radar. In other systems, the speed must be 
derived from at least two staggered position determinations. 
The object data, i.e. the sensor-specific descriptions of the 

10 objects, are used as input data for the object -classification 
system. The object of the classification is to group objects 
in above-mentioned classes. In the case of monitoring the 
surrounding field of the vehicle, these classes may include 
trucks, automobiles, motorcyclists, and pedestrians. For finer 

15 degrees, large and small automobiles or other classes such as 
station wagons, notchback cars, and SUV's are possible. Each 
class has specific characteristics, which describe the 
characteristic features of its own class or differences with 
respect to other classes. Such characteristics may be 

20 optionally organized in a database. Characteristics that 

describe an object may include the dimensions. In a further 
refinement, the possible speed range is then considered, as 
well. For example, if an object is moving at a speed of 75 
kilometers per hour, then based on this characteristic, it is 

25 highly improbable that it is a pedestrian. 

Figure 1 shows the device according to the present invention 
in a block diagram. An environmental sensor system 10 
transmits environmental sensor data to a signal processor 11, 
30 which sends the processed signals to a control unit 12. The 
control unit has a processor 13, which processes the sensor 
data and, using a memory that may be a database, classifies 
objects on the basis of the sensor data. Control unit 12 then 
transmits the classification result to other vehicle systems. 
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such as vehicle system 15 shown here by way of example, e.g. 
an occupant -restraint system. The action of vehicle system 15 
is determined on the basis of the object classification. 

5 Processor 13 ascertains the shape and the dimensions of the 
individual objects from the sensor data. In addition, the 
speed of the individual objects is ascertained from the sensor 
data. This may then be compared to a speed range. In this 
case, sensor system 10 is a video sensor system. However, it 
10 is also possible for a radar sensor system, e.g. a 77 

Gigahertz radar sensor system, to be used. An ultrasonic 
sensor system is also possible for the short range. 

One ascertains a three-dimensional point cloud of a physical 
15 object with the aid of a stereo video sensor. The points in 

this cloud are then combined (clustered) to form an object in 
the classification system. Therefore, the point cloud is 
described by a surrounding box (bounding box) , whose dimension 
corresponds to the dimensions of the physical objects. A 
2 0 further option for determining the dimensions is an analysis 

of the image. In this case, one may determine the edges of the 
vehicle and calculate the dimensions with the aid of the 
orientation (if known) . 

25 Figure 2 shows a flowchart of how an object is classified. In 
method step 2 00, sensor system 10 ascertains the objects in 
the surrounding field. Evaluating unit 11, which processes the 
signals for control unit 12, then generates the object data, 
which are transmitted to control unit 12, and consequently to 

30 processor 13. This is accomplished in method step 201. 

Processor 13 then carries out the object classification in 
method step 202. For this, it uses predefined, stored 
definitions of relevant object classes, i.e. on the basis of 
the shape, dimensions and speeds of the object. In this 
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context, the processor may also access a database 2 04 in which 
the class characteristics are stored, in order to carry out 
unequivocal classifications on the basis of the measuring 
results . 

The classification results are filtered as a function of time. 
This means that when an object has already been classified ten 
times as a small passenger car, for example, and should now be 
classified as either a small passenger car or possibly a 
pedestrian (same characteristics in this measurement) , the 
object continues to be classified as a small passenger car 
with the aid of the temporal filtering. There is also a reject 
class. All objects, which could not be assigned to any class 
(because they do not correspond to any characteristics of the 
classes in the model database), are categorized here. 

Depending on the application, a non-unique result may also be 
sufficient. For example, if an object was classified as either 
a large or a small vehicle and the application is designed for 
20 pedestrian protection, then this ambiguity is not relevant. 

In method step 2 05, the object classifications are then 
supplied to vehicle systems 15. In method step 206, the action 
of these vehicle systems 15 is then controlled as a function 
25 of the object classification. 

Figure 3 shows an example of a classification. Shown in 
picture 300 is a plan view of a corridor, which is monitored 
by a vehicle. Here, objects are classified by classifiers that 
3 0 are represented schematically. The objects are then covered by 
the rectangles and supplied one classifier each. Classifier 
302 shows a pedestrian. Classifier 303 shows a large passenger 
car. Classifier 314 shows the posts of a street lamp. 
Classifier 304 represents a small passenger car, as does 
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classifier 305 as well, while classifier 306, which is already 
in an outer monitoring region, represents a small truck. In 
addition, a delineator, a bicycle, a motorcycle, and a large 
truck may also be classified, as described in legend 307. 
5 Image 3 01 shows how a video camera in the vehicle monitors and 
classifies the scene. The pedestrian at the left edge of the 
image, who is denoted by classifier 3 08, additionally has an 
object orientation here, which, according to the legend, 
indicates that pedestrian 3 08 does not have an orientation. As 

10 is shown here, he or she is standing at the edge of the 
street, as well. Street lamp 309 also has this symbol 
indicating that it does not have an object orientation. 
Classification 310 shows a passenger car, which is traveling 
in front of the obseirver and is oriented in the front 

15 (forward) direction. Truck 311 next to it is also oriented in 
the forward direction, as is passenger car 311 and a further 
passenger car 313, as well. Other objects are not classified 
here. The object classification is also a function of 
distance. 
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